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[57] ABSTRACT 

An inner-product array processor is provided with 
thresholding of the inner product during each iteration 
to make more significant the inner product employed in 
estimating a vector to be used as the input vector for the 
next iteration. While stored vectors and estimated vec- 
tors are represented in bipolar binary (1,-1), only those 
elements of an initial partial input vector that are be- 
lieved to be common with those of a stored vector are 
represented in bipolar binary; the remaining elements of 
a partial input vector are set to 0. This mode of repre- 
sentation, in which the known elements of a partial 
input vector are in bipolar binary form and the remain- 
ing elements are set equal to 0, is referred to as trinary 
representation. The initial inner products correspond- 
ing to the partial input vector will then be equal to the 
number of known elements. Inner-product thresholding 
is applied to accelerate convergence and to avoid con- 
vergence to a negative input product. 

11 Claims, 7 Drawing Sheets 
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IMER-PRODUCT ARRAY PROCESSOR FOR 
RETRIEVAL OF STORED IMAGES 

REPRESENTED BY BIPOLAR BINARY ( + 1,-1) 
PIXELS USING PARTIAL INPUT TRINARY 5 
PIXELS REPRESENTED BY (+1,-1) 

ORIGIN OF THE INVENTION 

The invention described herein was made in the per- 
formance of work under a NASA contract, and is sub- 10 
ject to the provisions of Public Law 96-517 (35 USC 
202) in which the Contractor has elected to retain title. 

TECHNICAL FIELD 

This invention relates to associative recall of one or 15 
two dimensional vectors, sometimes more appropri- 
ately referred to herein as “images” or “image vectors”, 
from associative memory, and more particularly to an 
inner-product trinary associative memory in which 
image elements are stored in bipolar binary representa- 20 
tion (U— 1), and a partially known input image is in 
trinary neuron representation (1,0, — 1) in which known 
pixel elements are represented by bipolar binary values 
of 1 and — 1, and unknown elements are represented by 
0’s. 25 

BACKGROUND ART 

In many industrial and optical computing applica- 
tions, it is necessary to reconstruct an object’s complete 
description from a partial description. Systems for per- 30 
forming such image reconstruction are expected to have 
an a priori knowledge of the object image to be recon- 
structed. To fully utilize parallelism of optics, the mem- 
ory elements should be such that they can be accessed 
simultaneously and in parallel on the basis of data con- 35 
tent rather than by specific address or location. Such a 
memory, referred to as an associative memory, delivers, 
the output even when it is stimulated by a partial image 
input of the stored information. The mechanics of stor- 
ing, matching, and decision making determines the ac- 40 
tual performance of the associative memory. 

Hopfield has described a memory matrix that is easy 
to implement and requires very little time for learning 
and retrieval. J, J. Hopfield, “Neural networks and 
physical systems with emergent collective computa- 45 
tional abilities,” Proc. Natl. Acad. Sci. USA 79, 
2554-2558 (1982). The prior art reveals several imple- 
mentations of this model in which only bipolar binary 
numbers (1,-1) were used for storing vectors and uni- 
polar binary numbers (1,0) were used to address the 50 
memory. N. H. Farhat, D. Psaltis, A. Prata and E. Paek, 
“Optical implementation of the Hopfield model,” Appl. 
Opt. 24(10), 1469-1475 (1985) and B. Macukow and H. 

H. Arsenault, “Modification of the threshold condition 
for a content-addressable memory based on the Hop- 55 
field model,” Appl. Opt. 26(1), 34-36 (1986). Several 
different modifications to this initial work of Hopfield 
have already been proposed, and there have been sev- 
eral reported studies on various number representations 
of neurons. J. S. Bayley and M, A. Fiddy, “On the use 60 
of the Hopfield model for optical pattern recognition,” 
Opt. Commun. 64, 105-110 (1987) and M. Takeda and J. 

W. Goodman, “Neural networks for computation: num- 
ber representations and programming complexity,” 
Appl. Opt. 25(18), 3303-3046 (1986). 65 

The Hopfield model uses the outer product of a vec- 
tor with itself as a memory matrix for that vector and 
thereafter adds such memory matrices of each of the 


2 

nearly orthogonal stored vectors. An inner-product 
associative memory has been proposed based on Hop- 
field’s model (with autoneural interconnect). S. Y. 
Kung and H. K. Liu, “An optical inner-product array 
processor for associative retrieval,” Proc. SPIE, Vol. 
613, Nonliner Optics and Applications, 214-219 (1986). 
For matching and retrieval, the memory is addressed by 
an input vector V ,n . Accordingly, the ith element of the 
estimated vector V w is given by 

*?•- *"[*?, 

where \ p ^ m represents the pth element of the mth 
stored vector and n is the length of the vectors. 

The convergence mechanism of vectors in Hopfield’s 
neural network in relation to recognition of partially 
known patterns has been studied in terms of both inner 
products and Hamming distance, the measure of similar- 
ity between objects. In much of the prior art, the Ham- 
ming distance has been used as a measure of dissimilar- 
ity, overlooking the role of the inner products in deter- 
mining convergence. In cases other than that of the 
bipolar binary input, the Hamming distance criterion 
does not directly represent the. ongoing matching pro- 
cess of convergence. Inner-product weighting coeffici- 
ents play a more dominant role in many data representa- 
tions for determining the convergence mechanism. 

Each of the input representations, bipolar binary and 
unipolar binary, has its own merit. With bipolar binary 
input and thresholding, the convergence outcome of the 
associative memory always agrees with the minimum 
Hamming distance criterion. At the same time, the bipo- 
lar binary representation prevents the input from con- 
verging to a negative vector. On the other hand, the use 
of binary unipolar representation seems to recognize a 
partial input vector irrespective of its Hamming dis- 
tance. 

Recently it has been shown that in cases other than 
that of the bipolar binary input, the Hamming distance 
criterion does not directly represent the ongoing match- 
ing process of convergence. A. A. S. Awwal, M. A. 
Karim and H. K. Liu, “Associative memory: an opti- 
mum binary neuron representation,” Proc. SPIE, Vol. 
1053, Optical Pattern Recognition, 17-29, (1989). An 
object of this invention is to provide associative mem- 
ory using trinary neuron representation which over- 
comes these discrepancies. The resulting trinary asso- 
ciative memory may be used to process one dimensional 
as well as two-dimensional stored images, such as those 
encountered in industrial applications. 

STATEMENT OF THE INVENTION 

To combine the merits of both unipolar and bipolar 
neuron representations, an inner-product array proces- 
sor for associative retrieval of a best -match image with 
a partial image has been developed to make title initial 
inner product more significant. To do that, the stored 
images are represented by bipolar binary values of 1 and 

— 1, and only the elements of a partial image description 
, i.e., image pixels that are known (i.e., believed to be 
common with those of a stored image) are represented 
by bipolar binary values 1 and —1, and the remaining 
elements are set to 0. This mode of representation of a 
partial image description is referred to herein as trinary 
neuron representation, namely representation by 1 or 

— 1 for known pixel elements and 0 for all other ele- 
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ments not known. An inner product is formed for each 
trial comparison with stored images in an iterative 
search process for the closest estimated image. The 
initial inner products correspond to the known elements 
of the partial image description. After thresholding 
each of the inner products of each iteration, an esti- 
mated image is formed which is a linear combination of 
the bipolar binary stored image weighted by the thre- 
sholded inner products. Each of the estimated images is 
fed back as the input partial image description for the 
next iteration. In that manner, the maximum inner prod- 
ucts of each successive iteration are used as weights 
which generate an input vector for the next iteration. 
The process is repeated until convergence is reached. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a table which shows the combinations of the 
input and stored vectors that may appear in an inner 
product. 

FIG. 2a is a table of simulation results for input vec- 
tors whose M through n elements are identical to those 
of the stored vector ai while the remaining elements are 
zero for the particular stored vectors ai, where i = 1,2,3 
and 4. 

FIG. 2b is a table of simulation results for random 
input vectors. 

FIG. 3 is a table of simulation results for convergence 
of particular input vectors with starting weights calcu- 
lated for both trinary representation and unipolar repre- 
sentation. 

FIG. 4 illustrates two-dimensional images of four 
machine parts having reasonably similar Hamming dis- 
tances. 

FIG. 5 illustrates nine partial inputs used in evalua- 
tion of the present invention. 

FIG. 6 illustrates nine converged two-dimensional 
images corresponding to the nine partial inputs of FIG. 
5 for the stored images of FIG. 4 without inner-product 
thresholding when partial inputs are represented by 
unipolar binary values. 

FIG. 7 illustrates nine converged two-dimensional 
images corresponding to the partial input images of 
FIG. 5 without inner-product thresholding when par- 
tial inputs are represented by bipolar binary values. 

FIG. 8 illustrates nine converged two-dimensional 
images with inner-product thresholding corresponding 
to the partial input images of FIG. 5 when they are 
represented in trinary form. 

FIG. 9 is a block diagram of a two-dimensional, in- 
ner-product, trinary associative image system in accor- 
dance with the invention using bipolar representation 
for vector storage and trinary representation for partial 
input vectors, with thresholding of the inner products 
used for estimating a vector fed back as the input vector 
after each iteration until convergence is reached. 

DETAILED DESCRIPTION OF THE 
INVENTION 

Analysis 

The inner-product operation of the partial image 
input vector V /n with stored image vectors is equivalent 
to matching the input vector with all of the stored vec- 
tors. The “sgn” (i.e., hard limiter) operation in Equation 
(1) is basically a threshold operation that removes noise 
and clutter and retrieves the closest estimated vector. 
The estimated vector is then fed back as the new input 
vector. After repeated iterations, the estimated image 


4 

vector is expected to converge to the one stored image 
vector to which the initial input image is the closest. 

The estimated image vector can be expressed as a sum 
of weighted stored vectors, where the weights corre- 
5 spond to the inner products. Let a / represent the inner 
product of the ith stored image vector with the input 
image. Then the estimated image vector can be rewrit- 
ten as 


( M \ 

v" = sgn I 2 a/I'**’* J 

- sgn\a k V* tk + I, OiV«A 
15 \ / 

where a* > a, (when i= 1, . . . , n and i=£k) represents the 
largest inner product. One may safely assume that when 
the stored vectors are nearly orthogonal, the inner 
20 product between the input and one of the stored vectors 
will become maximum while those of the remaining 
stored vectors will be less than the maximum. The con- 
dition of orthogonality assures that as the iterated input 
vector moves closer to one of the stored vectors, it 
25 tends to move away from the remaining stored vectors. 

One very obvious method of retrieving the desired 
stored image vector is to choose by electronics and 
software the stored vector corresponding to a*. This 
approach is the same as correlating the input vector 
30 with all of the stored vectors by forming inner products 
and then choosing the one having the highest correla- 
tion. When the input image length of the vector is large 
compared with the number of stored images vectors, as 
is easily the case for a two-dimensional input image 
35 represented by N rows or rasters of M pixel elements 
per row which is thus represented by a vector having 
NM pixel elements, a* becomes significantly large com- 
pared with the a, (i.e., the remaining inner product), 
resulting in a correct convergence. 

40 

Neuron Representation and Decision Making 

The elements of each vector can be either unipolar 
binary (consisting of 1 and 0) or bipolar binary (consist- 
ing of 1 and — 1). A table set forth in FIG. 1 shows the 
45 combinations of the input and stored vector elements 
that may appear in the inner product for each of four 
cases. The vector element values within parenthesis are 
the actual values; the values outside parenthesis corre- 
spond to the bipolar value. Each of these four cases 
50 generates a particular type of inner product. This 
amounts to four different kinds of associative to repre- 
sentations of the inner product model since only it can 
provide insights into the convergence and retrieval 
processes. It may be noted that the nonzero autocon- 
55 nect (diagonal) case generally has a better convergence 
rate. K. F. Cheung, L. E. Atlas and R. J. Marks II, 
“Synchronous vs asynchronous behavior of HopfiekTs 
CAM neural net,” Appl. Opt. 26(22), 4808-4813 (1987); 
G. R. Gindi, A. F. Gmitro and K. Parthasarathy, “Hop- 
60 field model associative memory with non -zero -diagonal 
terms in memory matrix,” Appl. Opt. 27, 129-134 
(1988). 

When both the stored and input vectors are repre- 
sented in unipolar binary, the resultant inner product S 
65 is equal to the number of l’s that are present at identical 
positions of the two vectors. Note that the Hamming 
distance H is given by [n — (number of matching 
Ts-f number of matching 0’s)], where n is the number of 
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vector elements. The degree of similarity thus ranges 
from a minimum of 0 through a maximum of (n — H), 
with the expected value of the inner product being in 
the neighborhood of (n— H)/2. When the stored vector 
is bipolar, and the input vector is unipolar, the exact 5 
inner product is given by (S— d) , where d represents 
the number of elements equal to 1 in the input vector 
and the number of elements equal to — 1 (representing 
0) in the corresponding location of the stored vector. If 
50% of (n— H) vector element combinations corre- 10 
sponds to matching l’s and if 50% of H relates to the 
appearance of a 1 in the input vector while the corre- 
sponding stored value is —1, then the expected inner 
product is given by { [(n - H)/2] - (H/2)} = [n/2) - H]. 

The number of matching l’s was proposed as a de- 15 
gree of similarity for measuring the closeness of vectors. 

In general, the exact inner product is related to the 
Hamming distance, but the number of Vs in both stored 
and input vectors can heavily bias the inner products. 
This is why the input vector sometimes converges to a 20 
stored vector having a higher Hamming distance in- 
stead of converging to one having a smaller Hamming 
distance. Interestingly, the majority of the simulation 
models developed in the past have used Hamming dis- 
tance as a measure of dissimilarity and thus have failed 25 
to recognize the fact that the inner product plays a more 
important role in determining convergence. 

Hamming distance, however, can be incorporated in 
the model to directly affect the inner product of the 
vectors in an exact sense rather than in a probabilistic 30 
sense, only when the input vector representation is 
bipolar binary, as shown in FIG. 1 for case III. This is 
because the matching Vs as well as the matching 0’s 
contribute toward increasing the inner product. Thus, 
when both the input and stored vectors are bipolar 35 
binary, the exact inner product on each vector is 
(n— 2H). 

Consider next the case in which the stored vector is 
unipolar binary, while the input vector is bipolar binary, 
case IV in FIG. 1. Defining S' as the number of loca- 40 
tions having a — 1 at the input vector and a 1 at the 
corresponding location of the stored vector, the inner 
product is (S — S'), where S is the number of l’s present 
at identical positions of the stored and input vectors. If 
50% of H relates to the appearance of a — 1 (represent- 45 
ing 0) at the input while the corresponding stored value 
is 1 and 50% of (n — H) is matching 1 ’s, then the inner 
product is [(n/2) — H], 

For example, consider that only a portion of the 
stored vector is available as a partial input, in which 50 
case both matching l’s and matching 0’s should be al- 
lowed to contribute to the inner product since 0’s also 
carry information just as do l’s. In such a scenario, it 
may be desirable to represent both input and stored 
vectors in bipolar binary form, as in case III. Now con- 55 
sider the case of a complete portrait of the stored vec- 
tors with one or more false l’s in the input vector. It 
may then be desirable not to decrease the inner product 
owing to the presence of a false 1, but the presence of a 
false 0 in the input should reduce the inner product. 60 
Case IV of FIG. 1 achieves this particular objective. 

For measuring the degree of similarity, Hamming 
distance H can be used in only case III (bipolar-bipolar), 
while in all other cases the inner products directly rep- 
resent the degree of similarity. The Hamming distance 65 
works statistically but not exactly (since the matching 
0’s do not contribute to the inner product) for the other 
case, as will be shown by the simulation results. When 
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the input is random, the associative memory acts as a 
pattern classifier, wherein the search is for the stored 
vector that is its nearest neighbor. In this case, the Ham- 
ming distance serves well as a measure of closeness of 
the vector. Therefore, bipolar-bipolar representation is 
generally considered suitable for pattern classification. 
However, when there is a large Hamming distance 
between the input and one of the stored vectors (i.e., 
when 2H>n), the inner product may become negative. 
In the extreme case, a negative inner product may even 
lead to a convergence to a vector that is the comple- 
ment of the stored vector having the largest Hamming 
distance. 

The iterative method of searching in all cases begins 
to pose a problem when the inner products of the input 
vector with all stored vectors are so relatively close that 
no single inner product surpasses the others sufficiently 
to be recognized. The most disappointing problem, 
however, in the case of the unipolar binary input vector 
is that very often the highest inner product may not 
correspond to the lowest Hamming distance. Also, uni- 
polar binary inputs (having large Hamming distances) 
may result in complementary convergences. Although 
false convergences involving only unipolar binary in- 
puts have been recognized as problems in earlier works, 
the actual problem has not been either diagnosed or 
cured. Specific examples of vector convergences to the 
complement of a stored vector are identified and dis- 
cussed below. 

Inner-Product (IP) Thresholding 

As noted above, one would like to use a variable 
threshold in the inner product domain, such as average- 
inner-product thresholding, in order to choose the max- 
imum of the inner product. For Hamming distances Hi 
H 2 , . . . and H„, of the input vector from n stored vec- 
tors, the inner products are respectively (n — 2Hi), . . . 
(n — 2H„). When H,>n/2, the inner product becomes 
negative. With slight modification, for practical rea- 
sons, one could use zero-crossing thresholding in the 
inner product domain, which is reasonably easy to im- 
plement in hardware. If, for example, an input vector is 
more than n/2 Hamming distance away from any one of 
the stored vectors, then by scalar thresholding in the 
inner product domain one could prevent its conver- 
gence to the complement of the stored vector. From 
another point of view, zero-crossing thresholding is 
analogous to forgetting the vector(s) of very high Ham- 
ming distance, thus increasing the chance of correct 
convergence to the true stored vector on the theory 
that only the complement of a stored vector has any 
probability of having a large Hamming distance. This 
mimics a function in which an input stimulus does not 
compete with all of the stored information for locating 
its closest neighbor but competes only with a particular 
class of information, totally disregarding the rest of the 
classes. Consequently, the ith element of the estimated 
vector V es can be modified to 

r m / \~i (3) 

L ^ ■ ** *=1 *'*■* (ji 1 jj 

This is equivalent to introducing a nonlinear operation 
in the correlation domain. R. J. Marks II, L. E. Atlas, J. 
J. Choi, S. Oh, K. F. Cheung and D. C. Park, “Perfor- 
mance analysis of associative memories with nonlineari- 
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ties in the correlation domain,” Appl. Opt. 27(14), 
2900-2904 (1988); R. A. Athale, H. H. Szu and C. B. 
Friedlander, “Optical implementation of associative 
memory with controlled nonlinearity in the correlation 
domain,” Opt. Lett. 1 1, 482-484 (1986). 5 

Problem Diagnosis Using Simulation 

In the simulation to be described with reference to 
tables in FIG. 2a and FIG. 26, various types of input 
vectors (both random and nonrandom) were chosen. 10 
The four vectors that were chosen are as follows: 

al « (1 -111 -1111 -1111 -11 -1 -11 -1 -11) (4) 

a 2 = (-11 -111 -1 -11 -1 -1 -1 -111 -1 -1 - 111 ) 15 

a3 = (11 -1 -I -11 -111! -Ill -11111 -1 -1) 

o4 = (-nn -in -n -ii -i -i -n -mi -i> 

When an input vector is applied to the memory, it is 20 
actually related to all of the stored vectors rather than 
to just one stored vector. For example, an input vector 
that is at a Hamming distance x from the stored vector 
p may actually be closer to another stored vector q (at 
a Hamming distance of y) when x>y. Consequently, 25 
the process of accounting with only one Hamming 
distance is not sufficient. Therefore, the input vectors 
are identified by a set of four Hamming distances for the 
case under consideration of four vectors al through a4. 

For simplicity, ai(m:n) in the table of FIG. 2 a denotes 30 
a vector whose m through n elements are identical to 
those of the stored vector ai while the remaining ele- 
ments are zeroes. For comparison, a table set forth in 
FIG. 2b includes simulation results pertaining to several 
random inputs. For each of the input vectors in both 35 
FIG. la and FIG. 26, the corresponding Hamming 
distances and the inner products obtained immediately 
after the first iteration for both bipolar and unipolar 
representations are listed respectively under the second 
and third column headings. 40 

The entries in the remaining columns indicate to 
which stored vector the input vector converges, and 
how many iterations were required for convergence. 

FC entries indicate false convergences. In each column, 
the entry ai(j) indicates that the input vector converges 45 
to the stored vector ai after j iterations. Consider the 
first input vector al(l:7) in FIG. 2 a and the bipolar- 
bipolar format for the stored and input vectors. From 
the Hamming coordinate one can conclude that it is 
closest to al, and after three iterations when IP thresh- 50 
old is absent, it actually converges to al. The actual 
stable vector in this case prior to thresholding is 20al — 
6a2— 4a3 — 2a4 (not entered in the table) . When IP 
thresholding is present in accordance with the inven- 
tion, the vectors a2 and a3 are eliminated from the re- 55 
trieval process after the first iteration, and convergence 
is reached in two iterations, thus leading to a faster 
convergence with no ambiguity. With bipolar binary 
and unipolar binary representations, respectively, for 
the same stored and input vectors (first row of FIG. la), 60 
no convergence is achieved; instead, the output stabi- 
lizes to a false vector 6al — 10a2 + 6a3-f 2a4 (denoted in 
the table by FC), The initial inner products are (5, —3, 

— 1 , 3), but since 5 is less than the absolute sum of the 
other three inner products, these inner products drive 65 
al(l:7) toward a false vector. Obviously, IP threshold- 
ing would have favored al since a2 and a3 would have 
been out of consideration, and clearly 5al-f3a4 would 
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have converged to al, thus demonstrating another fea- 
ture of the invention, namely inner-product (IP) thre- 
sholding. Sometimes large negative inner products 
drive the vector to the complement of the vector (such 
as the stored vector a3 for the input vector al(l:4)). 
Such problems can be avoided by IP thresholding. 

Evaluation of Simulation Results 

By studing the findings of the tables in FIGS. 2a and 
26, one can make the following observations involving 
the convergence of the input vector to one of the stored 
vectors. 

First, the simulations support the assumption that in 
the case of unipolar binary representation for the input, 
the Hamming distance criterion does not determine the 
vector convergence since the input vector usually con- 
verges to a stored vector that corresponds to the high- 
est inner product. For example, the input vector a2(l:4) 
in the sixth row of FIG. 2a, when represented in unipo- 
lar binary form, incorrectly converges after three itera- 
tions to the stored vector a4 even though the Hamming 
distance criterion suggests that it should instead con- 
verge to a2. Such nonconvergence of input vectors to 
nonmininum Hamming distance vectors was also ob- 
served in an earlier work. B. L. Montgomery and B. V. 
K. Vijaya Kumar, “Evaluation of the use of the Hop- 
field neural network model as a nearest-neighbor algo- 
rithm,” Appi. Opt. 25, 3759-3766 (1986). 

Second, the input vectors using bipolar binary repre- 
sentation usually converge to a stored vector for which 
the inner product is the largest in magnitude. Whenever 
the largest inner product is negative, the convergence 
incorrectly leads to a vector that is the complement of 
the stored vector for which the negative inner product 
is maximum in the absence of thresholding. For exam- 
ple, in the absence of IP thresholding, the input vector 
al(l:4) in the fifth row, which is a derivative of al, 
incorrectly converges to — a3 instead of converging to 
al (as Hamming distance criterion would suggest). 
Clearly, this anomaly can be predicted by observing the 
inner product value —6 for the vector a3, while the 
inner product value corresponding to the vector al is 
only 2. Other simulations have also shown that when 
the input and stored vectors are both represented by 
bipolar binary numbers, convergence to complement 
vectors always occurs. H. K. Liu, T. H. Chao, J. Barhen 
and G. Bittner, “Inner-product optical neural process- 
ing and supervised learning,” Abstracts of the 1st Int. 
Neural Network Society Meeting, Vol. 1, P. 391 (Sept. 
1988). IP thresholding will avert convergence to com- 
plement vectors in this case. 

Third, when a partial stored vector is introduced as 
an input vector, it could actually be close to a different 
stored vector. For example, input vector a3(l:4) in the 
seventh row is closer to the stored vector a2 in terms of 
Hamming distance. With bipolar binary representation, 
this input vector converges to a2, which is correct but 
only with IP thresholding, which illustrates the advan- 
tage of IP thresholding in the case of bipolar binary 
representation. With unipolar binary representation, the 
input correctly converges to a3 with or without IP 
thresholding, which illustrates the superiority of using 
unipolr representation of the input vector with bipolar 
representation of the stored vectors. 
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Trinary Neuron Representation and Measure of 
Similarity 

From the observations of the simulation results in the 
preceding section, it is evident that each of the two 5 
(unipolar and bipolar) input representations has its own 
merit. With bipolar binary input and IP thresholding, 
the convergence outcome of the associative memory 
always agrees with the minimum Hamming distance 
criterion. At the same time, the bipolar binary represen- 10 
tation prevents the input from converging to a false 
(complement) vector. On the other hand, the use of 
binary unipolar representation seems to recognize a 
partial input irrespective of its Hamming distance. By 
examining the convergence mechanism and the inner 
product, the following is noted: 

(1) When the partial input vector is represented in 
unipolar binary, the largest inner product is usually 
contributed by those partial input vector elements 
known (i.e., nonzero elements) that are in common 20 
with corresponding elements of a stored vector. 

(2) The inner product corresponding to the unipolar 
binary input comes only from the matching Vs of 
the known elements. 

(3) When partial inputs are represented in bipolar 
binary, the largest inner product is contributed 
both by the known as well as the unknown ele- 
ments of the input vector since both the unknown 
elements and known 0’s are being represented by 
— 1. Consequently, known 0’s contribute toward 30 
increasing the inner products while the unknown 
elements contribute toward increasing/decreasing 
the inner products when the number of matching 
0’s is greater/less than the number of mismatched 
rs. 

(4) The advantage of having 0’s in the unknown por- 
tion of the partial inputs (represented in unipolar 
binary) is that the 0’s do not contribute toward 
either increasing or decreasing the inner product. 

To combine the merits of both unipolar and bipolar 40 
neuron representations, the present invention proposes 
the use of trinary neuron representation for the input 
vector, while the associative memory uses bipolar bi- 
nary representation for stored vectors. To make the 
inner product more significant, the new neuron repre- 45 
sentation makes use of matching 0’s, but to do that, only 
the elements of a partial input vector that are known 
(which will then be common with those of a stored 
vector) should be represented in bipolar binary, and the 
remaining elements should be set to zero. The initial 50 
inner product corresponding to the partial input vector 
will then be equal to the number of known elements. 
Thereafter, one can apply IP thresholding to accelerate 
convergence and to avoid convergence to a negative 
inner product, i.e., to a complement of the stored vec- 55 
tor. This mode of representation for the input vector, in 
which the known elements are in bipolar binary and the 
remaining elements are 0’s, is defined as trinary repre- 
sentation. 

The advantage of using trinary representation for an 
input vector may be readily appreciated from another 
point of view. To measure closeness, Hamming distance 
should be used only for the partial elements, and not for 
the whole vector, since it is understood that one does 
not know the remaining elements of the stored vector 65 
for a partial input vector. Consequently, there is no use 
comparing the rest of the stored vector with the un- 
known elements of the input vector. Thus, in the first 
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iteration, the unknown elements of the input vector are 
kept inert by setting them to zero. Then, only after the 
first iteration, the feedback vector is represented in 
bipolar binary for accelerating the convergence pro- 
cess. 

One-Dimensional Image Recognition with Trinary 
Representation 

All of the problematic cases of the table in FIG. la 
are now repeated with the proposed trinary representa- 
tion of the input vector and bipolar representation of 
stored vectors. These problem cases of the input vectors 
are, respectively, a3(l:4), a4(l:4), al(16:20), al(13:20) 
al(l 2,18:20) and a4( 1,2, 18:20). For simplicity, the de- 
15 tailed calculations for only one of the above cases, 
a3(l:4), are presented here. 

In the earlier representation, the input vector would 
have been represented as(l 100000000000000 
0 0 0 0) in unipolar and as (11-1-1— 1— 1— 1-1 
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 ) in 
bipolar binary, whereas in trinary representation, it is 
given by(l 1 — 1 — 10000000000000000), where 
the nonzero numbers in bipolar binary are known, and 
the trailing zeroes represent unknown elements of the 
25 input vector. 

Inner products after the first trial — ( — 2 0 4 -2). 

Inner products with IP thresholding = (0 0 4 0). 

Inner products after third iteration = (— 4 —6 20 

- 2 ). 

Inner products after IP thresholding =(0 0 20 0). 

35 Thresholding the inner products thus eliminates all but 
the maximum inner product. The inner products on the 
third trial exceeds the sum of absolute inner products of 
the remaining vectors, thus indicating that on the third 
trial the input has finally converged to a3. While only 
the case of a3(l :4) was worked out here, a table set forth 
in FIG. 3 lists the results of that and the other five cases. 

The trinary neuron representation works best with 
partial inputs, provided the positions of the partial input 
elements are known. However, from FIG. lb it is seen 
that when the input is totally random and it is not 
known which part of the input contains the correct 
information, bipolar binary representation for the inputs 
along with IP thresholding is preferable for achieving 
convergence to the expected stored vector having mini- 
mum Hamming distance, thus demonstrating the power 
of IP thresholding. 

Two-Dimensional Image Recognition with Trinary 
Representation 

Most visual information is two-dimensional. Accord- 
ingly, the performance of an associative memory with 
bipolar stored vectors and a trinary representation of 
input vectors will now be discussed in retrieving two- 
dimensional stored images. Consider that the associa- 
60 tive memory can store an array of mXn two-dimen- 
sional images, each in the form of a bipolar mn-element 
vector. 

FIG. 4 shows two-dimensional stored images of four 
machine objects, SI, S2, S3 and S4, each having 1 1 X20 
pixels. The set of Hamming distances for those stored 
images are (0,67,56,55), (67,0,77,94), (56,77,0,67) and 
(55,94,67,0), respectively. Therefore, the four stored 
images are reasonably similar as far as their relative 
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Hamming distances are concerned. FIG. 5 shows nine 
partial inputs PI through P9 used in evaluation of the 
trinary representation. Note that P2 and P6 are the 
partial inputs of the stored image (two-dimensional 
vector) SI; PI, P3, and P7 are the partial inputs of 5 
stored image (two-dimensional vector) S2; P4 and P8 
are the partial inputs of stored image (two-dimensional 
vector) S3; and P5 and P9 are the partial inputs of 
stored image (two-dimensional vector) S4. 

To compare the performances of different neuron 10 
representations, the partial inputs are encoded using 
unipolar binary, bipolar binary and trinary representa- 
tion. FIG. 6 shows nine retrieved images corresponding 
to the nine partial inputs when they are represented 
using unipolar binary representation. Similarly, FIGS. 7 15 
and 8 show the retrieved images corresponding to the 
partial inputs when they are represented using bipolar 
(1,-1) and trinary (1,0,— 1) representation, respec- 
tively. Note that IP thresholding has been used in the 
trinary case. The inner product is left unchanged when 20 
it is greater than or equal to the average of inner prod- 
ucts; otherwise it is set equal to zero. 

It can be seen that each of the trinary partial inputs 
readily converges to a stored two-dimensional vector to 25 
which it corresponds. From amongst the unipolar bi- 
nary partial inputs only PI and P3 converge to the 
stored two-dimensional vector S2, while the remaining 
partial inputs result in false convergences (indicated by 
FC). These FC images are different from the stored 
images, but they are formed from linear combinations of 
the stored images. Surprisingly, from amongst the bipo- 
lar binary partial inputs, none of the partial inputs leads 
to a correct convergence. Note, however, that the bipo- 
lar partial input vector P3 almost converges to stored 
vector S2. Further, there are two interesting cases that 
deserve special attention. The bipolar binary partial 
input vectors P5 and P9, both of which correspond to 
stored vector S4, do indeed converge to stored vector 
S4 after the first iteration, but with subsequent iterations ^ 
they converge to false vectors. 

The foregoing shows the effects of IP thresholding 
and of various input neuron representations, and show 
that an optimum neuron representation for effective 
associative recall is the trinary neuron representation, in 45 
which the known portion of the partial input is repre- 
sented in bipolar binary and the unknown portion is 
represented by 0’s. Trinary representation eliminates 
most of the discrepancies found in conventional neuron 
representations. 50 

The trinary neuron representation has been used to 
successfully retrieve from storage one-dimensional vec- 
tors as well as two-dimensional vectors of obscured 
machine objects using partial input vectors. Perfor- 
mance of trinary neuron representation makes the oth- 55 
erwise conventional bipolar associative memory a via- 
ble processor for error-free character recognition, All 
that is required is to use trinary representation for the 
partial input vector and to then feed back the initial 
inner product in bipolar representation corresponding 60 
to the partial input vector (ignoring the 0’s representing 
unknown elements). Thus, after the first trial, the inner 
product in bipolar representation is fed back as the 
partial input vector for each subsequent iteration until 
convergence is reached. Inner-product (IP) threshold- 65 
ing is employed to accelerate convergence and to avoid 
convergence to a negative inner product. The use of IP 
thresholding has been found to be advantageous even 
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with other forms of representation for stored and input 
vectors. 

A functional block diagram for a system is illustrated 
in FIG. 9 using a television camera 10 as a two-dimen- 
sional vector input source, and although the system may 
be readily implemented with electronic hardware under 
control of a programmed computer 11, it is preferably 
implemented with optical processing components for 
faster real time optical associative retrieval. An optical 
trinary associative memory is relatively convenient to 
implement using a polarization -encoded optical shad- 
ow-casting (POSC) system. M. A. Karim, A. A. S. 
Awwal, and A. K. Cherri, “Polarization-encoded opti- 
cal Shadowcasting logic units: design,” Appl. Opt. 26 
(14), 2720-2725 (1987) and A. A. S. Awwal and M. A. 
Karim, “Associative polarization-encoded optical 
shadow casting: gray-level image encoding for serial 
and parallel operations,” Appl. Opt. 28(2), 284-290 
(1989). Alternatively, a polarization -encoded hologram 
may be used. The POSC system has already been de- 
signed to accommodate mixed mode numbers for opti- 
cal computing applications. A. A. S. Awwal, M. A. 
Karim and A. K. Cherri, “Polarization -encoded optical 
shadow-casting scheme: design of multioutput trinary 
combinational logic units,” Appl. Opt. 26(22) 
4814-4818 (1987). The input vector can be represented 
by an array of light emitting diode (LED) source pat- 
terns, while the stored vectors can be coded on trans- 
parencies or in a liquid crystal spatial light modulator. 
H. K. Liu, J. A. Davis and R. A. Lilly, “Optical -data- 
processing properties of a liquid-crystal television spa- 
tial light modulator,” Opt. Lett. 10, 635-637 (1985) and 
H. K. Liu, S. Y. Kung and J. A. Davis, “Real-time 
optical associative retrieval technique,” Opt. Eng. 
25(7), 853-856 (1986). 

Referring to FIG. 9, the first function represented by 
block 12 is to quantize mxn pixels (elements) of a two- 
dimensional vector into bipolar binary values (1, — 1) for 
known elements and to set all unknown elements to 0. 
This function may be carried out by special purpose 
hardware or with just a frame store and an algorithm 
stored in the computer 11, i.e., by computer software. 
Once the initial input vector in trinary format has been 
fed into a search operation block 13, which performs 
the first (inner) multiplication of Equation (3), a switch 
14 is operated by the control computer 11 to connect 
the first search operation block 13 to accept each esti- 
mated vector V/« for the next and subsequent iterations 
until convergence is reached, after which the switch 14 
is returned to a position to accept a new input vector. 

The inner products of the first multiplication opera- 
tion in block 13 are processed through an IP threshold- 
ing block 15 (either zero-crossing or average of inner 
products). The IP thresholding function and the first 
multiplication function of the search operation may also 
be readily carried out by software stored in the com- 
puter 11, particularly if the stored bipolar vectors 16 are 
initially stored in computer memory, such as by pro- 
cessing images of known, unobscured objects through 
the quantizing block 12 also implemented with com- 
puter software. Note that if an unobscured object is 
scanned by the television camera to store a vector, 
quantizing the pixels to 1 and — 1 values results in bipo- 
lar binary representation, there being no unknown pix- 
els to be set to 0. 

A weighting function (second multiplication) is car- 
ried out in block 17. That multiplication function, 
shown within brackets in Equation (3), may also be 
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readily carried out in the computer 11 by software. 
Then the products of this second multiplication (the 
outer products of Equation (3)) are thresholded in block 
18 to carry out the function of the outer M sgn” in Equa- 
tion (3) and to format the next input vector Vj* pres- 5 
ented through switch 14 in bipolar binary. Thus, after 
thresholding each of the inner products of each iteration 
in block 15, blocks 17 and 18 form an estimated vector 
which is a linear combination of the bipolar binary 
stored vector weighted by the thresholded inner prod- 1° 
ucts. Each estimated vector is fed back as the input 
vector for the next iteration until convergence is 
reached. Note that the estimated vectors are also fed to 
the control computer 11 which is programmed to deter- 
mine when convergence has been reached. 15 

The functional block diagram of FIG. 9 could, of 
course, be implemented with special purpose hardware, 
particularly optical hardware, for greater speed where 
real-time recognition of input vectors requires such 
greater speed than, for example, recognition of machine 20 
parts on a conveyor belt. But the simulation results 
presented above have been successfully carried out 
with computer software at speeds that would likely be 
comparable to conveyor belt speeds. Consequently, it is ^ 
intended that the claims be interpreted to cover imple- 
mentation with electronic hardware and optical hard- 
ware as well as software. 

What is claimed is: 

1. In an inner-product array processor system for 3Q 
associative retrieval of a stored image from among a 
plurality of different images stored in a digital memory 
using a partial image input, each pixel elements of said 
stored images being represented by bipolar binary val- 
ues of -f 1 and — 1, where a + 1 is represented by a fixed 35 
positive value of voltage and a — 1 is represented by a 
fixed negative value of voltage, a method comprised of 

representing each known pixel element of said partial 
image by said bipolar binary values of -f- 1 and — 1, 
and setting the value of each unknown pixel ele- ^ 
ment of said partial image to 0, where a 0 is repre- 
sented in said system by a zero voltage, 
forming an inner product for each trial comparison 
with stored images in an iterative search process 
for the closest estimated image with the initial inner 45 
products corresponding to the known pixel ele- 
ments of the partial image input, 
forming an estimated image which is a linear combi- 
nation of the bipolar binary stored image weighted 
by said inner products, 50 

feeding back each of said estimated images repre- 
sented in bipolar binary values of + 1 and — 1 as the 
input image for the next iteration, thereby using the 
maximum inner products of each successive itera- 
tion as weights which generate an input image for 55 
the next iteration, and 

repeating the process until convergence is reached, 
thereby retrieving a stored image from said mem- 
ory which corresponds to a best match of a stored 
image with said partial image input. 60 

2 . A method as defined in claim 1 including the step 
of thresholding each of said inner products formed. 

3. A method as defined in claim 2 wherein said step of 

inner product thresholding is comprised of average- 
inner-product thresholding. 65 

4. A method as defined in claim 2 wherein said step of 
inner-product thresholding is comprised of zero-cross- 
ing thresholding. 
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5. In an inner-product array processor system for 
associative retrieval of a stored image represented by 
bipolar pixel elements from among a plurality of differ- 
ent stored images, wherein pixel elements of each image 
are stored in a digital memory using bipolar voltage 
values of 1 and — 1, an iterative method of finding a 
best match of a stored image with a partial input image 
having known pixel elements represented by bipolar 
values of -M and — 1, and unknown pixels represented 
by zero voltage value by finding the maximum inner 
product of said partial input image and each of said 
stored images to be fed back as an input image for each 
iteration after the first, said iterative method comprising 
thresholding of the inner products for finding a maxi- 
mum inner product from among all inner products dur- 
ing each trial search, and using said maximum inner 
product as a partial input image for a subsequent itera- 
tion until convergence is reached thereby retrieving a 
stored image from said memory which corresponds to a 
best match of said partial input image with a stored 
image. 

6. In an inner-product array processor system for 
associative retrieval of a stored image, a method as 
defined in claim 5 wherein said inner-product threshold- 
ing is accomplished by average inner-product thre- 
sholding. 

7. In an inner-product array processor system for 
associative retrieval of a stored image, a method as 
defined in claim 5 wherein said inner-product threshold- 
ing is accomplished by zero-crossing thresholding. 

8. In an inner-product array processor for associative 
retrieval of a stored image from among a plurality of 
different images stored in memory, a system comprising 

means for storing a plurality of images as pixel ele- 
ments represented by bipolar binary values of 4- 1 
and —1, where a +1 is represented by a fixed 
positive value of voltage and a 31 1 is represented 
by a fixed negative value of voltage, 

means for representing a partial binary input image to 
be compared with said stored images in search of a 
best match with a stored image, said representation 
being by bipolar binary values of + 1 and — 1 par- 
tial input image, where 0 is represented in said 
system by a zero voltage, 

means for forming an inner product for each trial 
comparison with stored images in an iterative 
search process for the closest estimated image until 
convergence is reached, 

means for estimating an image which is a linear com- 
bination of a bipolar stored image for each iteration 
of said search process and weighted by said inner 
products, and 

means for feeding back said estimated image after 
each iterative trial as an input image represented in 
bipolar binary values +1 and —1. 

9. In an inner-product array processor, a system as 
defined in claim 8 including means for inner product 
thresholding to find the maximum inner product to be 
used to generate an estimated image fed back as the net 
input image after each iterative trial. 

10. In an inner-product array processor, a system as 
defined in claim 9 wherein said inner product threshold- 
ing is accomplished by average-inner-product thre- 
sholding. 

11. In an inner-product array processor, a system as 
defined in claim 9 wherein said inner product threshold- 
ing is comprised of a zero-crossing thresholding. 
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